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FOREWORD 


This  research  was  conducted  within  the  in-house  independent  laboratory  research 
program  under  a  project  entitled  Cognitive  Speed  and  Learning  Potential  (PE61152N, 
ZROOO-0 1-042.003).  During  FY84  the  work  has  transitioned  to  Future  Technologies  for 
Manpower  and  Personnel  (PE62763,  RF63- 52 1-806).  The  primary  objectives  of  this  effort 
were  to  improve  the  Navy's  ability  to  select  technically  trainable  personnel  from  a 
dwindling  manpower  pool  and  to  determine  whether  speed  of  information  processing  is 
related  to  other  kinds  of  intellectual  aptitude. 

This  report  describes  a  first  attempt  by  the  Navy  Personnel  Research  and  Develop¬ 
ment  Center  to  investigate  the  relationship  between  cognitive  speed  and  success  in  a 
technical  training  program.  The  battery  of  cognitive  speed  tests  will  be  further  developed 
and  evaluated  in  upcoming  research  in  both  military  and  civilian  settings. 


3.  W.  RENARD  3.  W.  TWEEDDALE 

Captain,  U.S.  Navy  Technical  Director 

Commanding  Officer 


SUMMARY 


Problem 


There  is  a  longstanding,  widely  recognized  need  for  methods  of  testing  aptitude  that 
are  fast,  easy  to  administer,  culture-free,  and  predictive  of  meaningful  criteria,  such  as 
achievement  in  or  graduation  from  training.  Current  military  testing  programs  employing 
conventional  paper -and-pencil  tests  appear  to  have  reached  the  upper  limit  of  their  utility 
many  years  ago.  Recent  developments  in  the  field  of  cognitive  science  and  in  computer 
applications  to  testing  offer  considerable  promise  for  improving  the  technology  for 
discovering  and  measuring  an  individual's  talents  and  aptitudes. 

Purpose 

The  purpose  of  the  present  effort  is  to  develop  computer-presented  tests  of  cognitive 
speed  and  to  assess  their  effectiveness  as  predictors  of  performance.  Two  hypothesized 
measures  of  cognitive  speed--inspection  time  (IT)  and  reaction  time  (RT) — were  in¬ 
vestigated. 

Approach 

A  battery  of  cognitive  speed  tests  was  programmed  for  administration  via  micro¬ 
computer  and  given  to  155  entering  students  at  the  basic  electricity  and  electronics 
(BE&E)  school,  Naval  Training  Center,  San  Diego.  The  tests  measured  (1)  reaction  time, 
the  time  in  milliseconds  it  takes  a  subject  to  respond  to  simple  and  complex  stimuli,  and 
(2)  inspection  time,  the  shortest  time,  in  milliseconds,  a  subject  needs  to  discriminate 
which  of  two  briefly  presented  lines  is  longer. 

Results 


Significant  relationships  were  found  between  scores  on  several  of  the  cognitive  speed 
tests  and  rate  of  progress  in  the  self-paced  BE&E  course.  The  results  support  the 
continued  investigation  of  cognitive  speed  as  a  potentially  useful  predictor  of  technical 
school  performance. 
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INTRODUCTION 


Problem 


There  is  a  longstanding,  widely  recognized  need  for  improved  methods  of  testing 
aptitude  that  are  fast,  easy  to  administer,  culture-free,  and  predictive  of  meaningful 
criteria,  such  as  achievement  in  or  graduation  from  training.  While  current  military 
testing  programs  employing  conventional  paper-and-pencil  tests  appear  to  have  reached 
the  upper  limit  of  their  utility,  recent  developments  in  the  field  of  cognitive  science  and 
in  computer  applications  to  testing  may  provide  alternative  means  for  discovering  and 
measuring  an  individual's  talents  and  aptitudes.  One  approach  currently  being  pursued  is 
the  investigation  of  individual  differences  in  basic  information  processing  skills  (e.g., 
Hunt,  1983;  Sternberg,  1981).  In  particular,  some  scientists  have  reported  that  simple 
tests  that  provide  an  index  of  how  quickly  and  accurately  a  person  can  perceive  a  simple 
stimulus  may  yield  a  surprising  amount  of  information  about  that  person's  underlying 
mental  capacity  (e.g.,  Brand  Sc  Deary,  1982). 

Purpose 

The  purpose  of  the  present  effort  is  to  develop  computer-presented  tests  of  cognitive 
speed  and  to  assess  their  effectiveness  as  predictors  of  performance.  Two  hypothesized 
measures  of  cognitive  speed — inspection  time  (IT)  and  reaction  time  (RT) — were  in¬ 
vestigated. 

Background 

IT  tests  generally  require  the  subject  to  discriminate  between  rapidly  presented 
stimuli,  usually  lines.  For  example,  a  subject  might  be  shown  two  horizontal  lines  of 
unequal  length  and  asked  whether  the  longer  line  was  on  the  right  or  left.  The  duration  of 
time  the  subject  sees  the  lines  is  gradually  reduced  until  his  or  her  threshold  is  found,  with 
threshold  being  defined  as  the  duration  in  milliseconds  at  which  the  subject's  judgment 
would  be  correct  on  a  predetermined  percentage,  say  90  percent,  of  the  trials.  An 
analogous  series  of  trials  using  vertical  lines  is  sometimes  employed  along  with  or  in  place 
of  horizontal  lines.  The  use  of  forward  and  backward  masks—long  lines  that  the  subject 
perceives  as  covering  the  two  stimulus  lines  before  and  after  they  are  presented — ensures 
that  responses  are  based  on  information  that  has  actually  been  encoded  rather  than  on 
afterimages  or  other  peripheral  (as  opposed  to  central)  processes.  The  subject's  threshold 
value  is  therefore  presumed  to  be  largely  a  function  of  his/her  speed  of  information 
processing. 

Experimental  studies  using  IT  procedures  have  found  significant  correlations  between 
IT  and  more  traditional  measures  of  intelligence  (IQ)  (e.g.,  Nettlebeck  Sc  Lally,  1976; 
Brand,  in  press),  and  between  IT  and  the  ability  to  perform  certain  mechanical  tasks  (e.g., 
Nettlebeck,  Chesire,  Sc  Lally,  1979).  Also,  analogous  research  involving  speed  of 
processing  auditory  stimuli  has  demonstrated  a  positive  relationship  to  aptitude  (e.g.,  Raz, 
Willerman,  Ingmundson,  Sc  Hanlon,  1983).  However,  many  of  the  studies  suffer  from 
methodological  shortcomings  (Irwin,  1984),  and  there  is  also  controversy  concerning  the 
range  of  IQs  over  which  the  IQ/IT  relationship  holds.  A  recent  review  by  Nettlebeck 
(1982)  suggests  that  many  of  the  reported  correlations  between  IT  and  IQ  measures  may 
have  been  inflated  by  including  subjects  having  lower  than  average  intelligence;  however, 
other  authors  (e.g.,  Brand  Sc  Deary,  1982)  maintain  that  the  finding  of  a  strong 
relationship  between  IT  and  IQ  does  not  appear  to  be  critically  dependent  on  the  inclusion 
of  low  IQ  subjects. 


Research  using  RT  procedures  dating  back  to  the  turn  of  the  century  has  been 
regarded  as  demonstrating  little  relationship  between  RT  and  intelligence  measures. 
However,  recent  advances  in  electronic  technology  have  permitted  time  to  be  measured 
more  precisely,  and,  more  important,  RT  (decision-making)  to  be  separated  from 
movement  time  (MT)  (muscle  response);  accordingly,  this  view  has  been  revised.  Several 
investigators  (e.g.,  Carlson  Sc  Jensen,  19S2;  Jensen,  1980;  Vernon,  1983)  have  found 
promising  results  using  RT  measures.  These  researchers  have  reported  that  the  linear 
increase  in  RT  to  visual  or  auditory  stimuli  as  a  function  of  the  amount  of  information 
presented  is  negatively  correlated  with  IQ;  that  is,  the  brighter  the  subject,  the  less  his  or 
her  reaction  time  is  slowed  by  presenting  more  complicated  stimuli. 

The  present  study  is  a  first  attempt  by  this  Center  to  examine  whether  IT  and  RT 
measures  provide  useful  data,  given  the  range  of  aptitude  found  in  military  populations. 
Further,  by  analyzing  the  experimental  IT  and  RT  measures  in  conjunction  with  scores 
from  traditional  tests,  it  is  hoped  the  role  of  cognitive  speed  in  mental  functioning  will  be 
better  understood. 


APPROACH 


Criteria 


The  Navy's  basic  electricity  and  electronic  (BE&E)  preparatory  schools  provide 
students  with  technical  knowledge  and  skills  necessary  for  success  in  their  follow-on  "A" 
schools,  which,  in  turn,  provide  training  for  specialized  positions  in  the  Navy.  Therefore, 
predicting  progress  in  and  successful  completion  of  BE&E  school  is  an  important  means  of 
determining,  at  minimum,  short-term  performance  in  the  Navy. 

The  BE&E  schools  employ  a  self-paced  computer-managed  instruction  system. 
Students  must  demonstrate  100  percent  mastery  on  a  given  module  before  being  allowed 
to  proceed  to  new  material. 

For  purposes  of  this  study,  two  performance  measures  in  BEScE  school  were  used  as 
the  criteria;  (1)  score  obtained  (percentage  of  items  correct)  on  the  comprehensive  exam 
given  on  modules  20-25,  and  (2)  time  to  learn  (TTL);  that  is,  the  time  the  student  needed 
to  complete  the  first  25  modules.  Module  25  was  used  as  a  cut-off  since  only  a  small 
number  of  subjects  received  instruction  beyond  this  point.  TTL  is  regarded  as  being  a 
sensitive  and  reliable  measure  of  individual  differences  (Gettinger,  1984). 

Other  data  available  included  the  subject’s  scores  on  the  tests  of  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB).  In  addition,  information  was  obtained  from  the 
subjects  regarding  their  vision,  state  of  fatigue,  smoking  habits,  and  handedness. 

Procedure 


A  total  of  155  incoming  students  at  the  BE&E  School  at  the  Naval  Training  Center, 
San  Diego,  were  administered  a  battery  of  IT  and  RT  tests  between  January  and  March  of 
1983.  The  battery  took  approximately  1  hour  to  administer  on  a  TRS-80  microcomputer 
and  comprised  the  following; 

1.  Three  IT  tests  with  horizontal  lines  on  single  planes; 

a.  Dashed  lines  (12.7mm  and  15.9mm  long),  dashed  mask. 
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b.  Dashed  lines  (12.7mm  and  15.9mm  long),  solid  mask. 

c.  Solid  lines  (12.7mm  and  15.9mm  long),  solid  mask. 

2.  Three  IT  tests  with  horizontal  lines  on  separate  planes: 

a.  Dashed  lines  (12.7mm  and  15.9mm  long),  dashed  mask. 

b.  Dashed  lines  (12.7mm  and  15.9mm  long),  solid  mask. 

c.  Solid  lines  (12.7mm  and  15.9mm  long),  solid  mask. 

3.  Two  IT  tests  with  vertical  lines: 

a.  Dashed  lines  (33.7mm  and  44.5mm  long),  dashed  mask. 

b.  Solid  lines  (19mm  and  22.2mm  long),  solid  mask. 

4.  One  RT  test  with  one-,  three-,  or  five-key  choice  tasks. 

The  IT  graphic  displays  were  created  using  the  TRS-80  character  set  and  the  POKE 
command  in  TRS-80  BASIC.  Line  lengths  differed  for  the  vertical  and  horizontal  displays 
due  to  the  limited  graphic  capabilities  of  the  apparatus.1  All  tests  were  adaptive;  that  is, 
they  were  made  easier  or  harder  in  response  to  the  subject's  performance.  Subjects 
responded  by  pressing  specified  keys  on  the  TRS-80  keyboard.  The  initial  display 
durations  were  relatively  long  in  order  to  build  subject  confidence  through  a  series  of 
correct  responses.  In  the  disp'ay  duration  algorithm,  if  the  subject  made  two  consecutive 
correct  answers,  he  or  she  was  allowed  to  attempt  the  next  higher  level  of  difficulty. 
However,  if  the  subject  made  an  error,  the  test  decreased  in  difficulty  and  the  rate  at 
which  the  test  difficulty  converged  was  reduced  (i.e.,  size  of  the  variations  in  stimulus 
duration  was  decreased).  The  briefest  display  durations  were  approximately  40  milli¬ 
seconds  (msec). 

In  the  RT  test,  subjects  were  presented  with  one,  three,  or  five  open  squares  on  the 
screen  and  instructed  to  press  the  appropriate  numbered  key  as  quickly  as  possible  after  a 
square  was  filled  in  (see  Figure  1).  At  the  beginning  of  each  trial,  the  subject  held  down 
the  space  bar  until  one  of  the  stimulus  squares  was  filled  in.  His  RT  was  the  number  of 
msec  between  the  onset  of  the  stimulus  (square  filled  in)  and  the  instant  he  released  the 
space  bar.  His  movement  time  (MT)  was  the  number  of  msecs  between  release  of  the 
space  bar  and  the  pressing  of  the  correctly  numbered  response  key. 

Before  each  test,  students  were  given  instructions  and  several  practice  trials,  until  it 
was  clear  that  they  understood  the  task.  After  each  presentation,  they  were  provided 
with  performance  feedback  via  computer  display.  If,  on  an  IT  test,  a  subject  made  two 
consecutive  correct  responses  at  the  highest  level  of  difficulty,  the  test  was  terminated 
and  the  subject  was  assigned  a  score  of  "1."  (The  scale  the  subjects  saw  was  reversed, 
with  1000  being  the  highest  possible  score.)  Otherwise,  a  subject  was  tested  until  he  had 
made  1 1  errors  on  each  IT  test;  his  or  her  score  was  based  on  the  final  display  speed. 


‘The  TRS-80  was  selected,  despite  its  limitations,  because  an  extensive  library  of 
relevant  software,  including  reaction  time  and  inspection  time,  were  commercially 
available  for  the  TRS-80.  However,  these  programs  proved  unsuitable  and  new  ones  were 
written. 
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a.  Subject  is  told  to  get  ready 
to  respond. 


b.  As  soon  as  a  square  (6  above)  is  filled  in, 
the  subject  must  press  the  corresponding 
key  (#6)  as  rapidly  as  possible. 


Figure  1.  Example  RT  test  with  five-choice  task. 

Analysis 

Zero-order  and  multiple  correlations  were  computed  among  the  cognitive  speec 
measures  and  the  school  performance  criteria. 


RESULTS 

Since  this  was  an  exploratory  study,  the  intention  was  to  develop  a  variety  of  IT  tests 
to  investigate  display  characteristics  (i.e.,  dashed  or  solid  lines,  vertical  or  horizontal,  one 
or  two  planes)  that  might  increase  test  effectiveness.  The  students  experienced  no 
trouble  in  understanding  the  instructions  and  most  seemed  to  enjoy  their  participation. 
The  students  proved  to  be  more  able  than  anticipated  and  on  one  test  (lc),  over  one  half 
achieved  the  highest  possible  score. 

An  additional  problem  involved  loss  of  subjects  due  to  conflicting  demands  on  the 
students'  time.  Although  all  155  subjects  took  the  RT  test,  not  all  completed  the  IT  part 
of  the  test  battery.  Also,  preliminary  analysis  indicated  that  the  female  members  of  the 
sample  (N  =  11),  had  significantly  slower  reaction  and  movement  times  under  all  three  RT 
conditions.  Further,  the  relationships  between  the  RT  measures  obtained  by  the  females 
and  the  school  criterion  variables  were  in  the  opposite  direction  than  those  for  males.  It 
was  decided  that  the  final  sample  should  include  only  the  male  subjects  who  took  the 
entire  test  battery  (N  =  112). 


There  were  no  significant  correlations  between  performance  on  the  cognitive  speed 
measures  and  corrected  vision,  near/far  sightedness,  or  self-reported  fatigue.  Chi-square 
analyses  indicated  no  significant  effects  attributable  to  test  administrator,  race,  or 
handedness. 

One-,  three-,  and  five-key  reaction  and  movement  times  were  selected  as  the  best 
predictors,  along  with  IT  tests  lb,  2c,  and  3a  because  (1)  they  had  the  highest  correlations 
with  school  progress  and  (2)  they  represented  different  types  of  stimulus  displays. 

The  distribution  statistics  for  the  criterion  measures  are  presented  in  Table  1.  Both 
measures  are  distributed  with  acceptable  symmetry,  making  transformations  unnecessary. 


Table  1 

BEdcE  School  Criterion  Measures 


Standard 

Mean 

Median 

Deviation 

Comprehensive  exam 
score  (%  correct) 

85.3 

84.1 

6.99 

Time  to  complete  the 
first  25  modules  (hours) 

325.0 

312.0 

79.69 

Table  2  presents  the  intercorrelations  between  the  cognitive  speed  measures. 
Although  most  of  the  RT  and  IT  tests  are  significantly  related  to  one  another,  some  of  the 
correlations,  particularly  those  for  IT  tests,  are  quite  low.  The  intercorrelation  between 
the  school  performance  criteria— time  required  to  complete  the  first  25  modules  (TIME 
25)  and  the  score  on  the  module  25  exam  (COMP  25)— was  -.43  (p  <  .001). 

Table  2  also  presents  the  correlations  between  the  cognitive  speed  measures  and  the 
school  performance  criteria.  The  best  predictors  from  among  the  RT  measures  were  5- 
key  RT  and,  unexpectedly,  1-key  movement  time.  For  the  IT  tests,  a  composite  measure 
based  on  the  subject's  average  z-score  for  IT  tests  lb,  2c,  and  3a  was  the  best  predictor  of 
school  success,  with  significant  relationships  to  both  performance  criteria.  Combining  the 
5-key  RT  and  the  composite  IT  score  yielded  a  multiple  R  of  .53  (p  <  .01)  in  predicting 
time  to  complete  modules  1-25.  It  would  be  of  interest  to  compare  the  cognitive  speed 
tests  with  the  tests  used  operationally  in  selecting  and  assigning  students  to  the  BE&E 
school;  however,  for  several  reasons,  there  is  no  statistically  valid  way  of  making  such  a 
comparison.  One  problem  resides  in  the  use  of  a  variety  of  test  composites  in  assigning 
students  to  the  school,  depending  on  what  naval  rating  they  will  be  assigned  after 
graduation.  Nevertheless,  as  a  rough  approximation  of  operational  test  validity,  scores  on 
the  Armed  Forces  Qualification  Test  (AFQT)  were  correlated  with  time  to  learn,  the 
result  being  a  validity  coefficient  of  .38.  Because  the  AFQT  subtests  incurred  some 
restriction  in  range,  having  been  used  to  some  extent  in  selecting  students,  and  because 
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the  .53  value  for  the  cognitive  speed  tests  represents  an  unshrunken  multiple  R,  the 
cognitive  speed  and  AFQT  tests  may  be  similar  in  validity  for  the  present  population. 

None  of  the  experimental  measures  were  predictive  of  academic  disenrollment. 

DISCIT5ION  AND  CONCLUSIONS 

As  a  preliminary  attempt  to  adapt  IT  methodology  to  microcomputers,  the  present 
study  provided  mixed  results.  The  problem  with  many  of  the  experimental  tests  being  too 
easy  demonstrates  the  need  for  an  apparatus  capable  of  extremely  brief  display  durations. 
The  fact  that  the  intercorrelations  between  the  tests  themselves  were  low  is  also  cause 
for  concern,  although  this  may  be  partly  due  to  the  presence  of  a  ceiling  effect  on  some 
of  the  tests.  Follow-on  research  with  higher  capability  instrumentation  has  been  initiated 
to  further  clarify  this  issue. 

The  data  for  RT  also  pose  a  problem.  The  finding  that  MT  was  sometimes  a  better 
predictor  of  performance  than  RT  was  unexpected.  However,  it  should  be  noted  that  the 
correlation  between  RT  and  school  performance  increased  as  the  RT  task  increased  in 
complexity,  implying  that  raising  the  number  of  response  options  beyond  the  maximum  of 
five  used  in  this  study  may  result  in  a  stronger  link  between  RT  and  school  performance. 
The  inverse  relationship  between  RT  and  school  performance  among  females  was  also 
unexpected  and  does  not  appear  to  have  been  reported  elsewhere  in  the  literature. 
However,  because  of  the  small  number  of  women  in  the  sample  it  probably  is  unwise  to 
draw  conclusions  without  further  data. 

As  indicated  previously,  a  controversy  exists  concerning  the  relationship  between 
cognitive  speed  and  more  traditional  measures  of  intelligence  (e.g.,  Nettlebeck  &  Kirby, 
1983;  Hulme  &  Turnbull,  1983).  Nevertheless,  the  findings  of  the  present  study  suggest 
that  the  experimental  cognitive  speed  tests,  if  presented  on  a  higher  capability  apparatus, 
might  make  a  significant  contribution  to  the  prediction  of  training  performance.  How¬ 
ever,  many  issues  remain  to  be  clarified,  including  the  optimal  length,  configuration,  and 
orientation  of  the  IT  display  lines,  the  fairness  of  RT  and  IT  tests  for  female  subjects,  and 
the  exact  nature  of  the  relationship  between  cognitive  speed  measures  and  more 
traditional  indices  of  aptitude.  Follow-on  work  is  expected  to  shed  more  light  on  these 
and  other  aspects  of  cognitive  speed  research. 


2 When  the  relationships  between  the  5-key  RT  test,  the  composite  IT  measure,  and 
the  ASVAB  test  scores  were  examined,  IT  was  found  to  be  most  strongly  related  to  the 
ASVAB  Mechanical  Comprehension  and  Auto  and  Shop  Information  tests  (r  =  .35  and  .33), 
while  RT  correlated  with  the  Attention  to  Detail  and  Auto  and  Shop  Information  tests  (r  = 
.53  and  .32).  Inspection  and  reaction  time  generally  were  not  related  to  ASVAB  tests 
used  to  measure  general  intellectual  ability  (e.g.,  Arithmetic  Reasoning  and  Word 
Knowledge). 


>>  -j.  V. 
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